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Software  Engineering  Process  Groups: 
Resuits  of  the  1992  SEPG  Workshop  Event 
Evaiuation  and  a  First  Report  on  SEPG  Status 


Abstract:  This  report  contains  a  summary  of  the  results  from  a  questionnaire 
administered  to  participants  in  the  Sof^are  Engineering  Process  Group 
(SEPG)  Workshop  held  in  Tysons  Comer,  Virginia,  in  April  of  1992.  The 
purpose  of  the  questionnaire  was  twofold:  (1)  to  ask  the  participants  for  their 
judgments  about  the  quality  of  the  week-long  event,  and  (2)  to  begin  collecting 
information  comparing  the  experiences  of  existing  SEPGs.  The  participants 
reported  a  generally  high  degree  of  satisfaction  with  the  content  and  conduct 
of  the  event.  Although  descriptions  about  SEPG  characteristics  and  activities 
apply  only  to  the  organizations  whose  representatives  attended  the  workshop, 
our  findings  suggest  recent  and  rapid  growth  in  the  SEPG  community.  Of  the 
1 69  responding  participants,  72  percent  stated  that  their  organizations  have 
SEPGs,  and  over  three-quarters  of  the  SEPGs  have  been  established  since 
1990. 


1  Introduction 

On  April  8  and  9,  the  Washington,  D.C.,  Software  Process  Improvement  Network  (SPIN)  and 
the  Software  Engineering  Institute  (SEI)  jointly  sponsored  the  1992  Software  Engineering 
Process  Group  (SEPG)  Workshop.  The  workshop  was  held  in  conjunction  with  the  SEI 
sponsored  Capability  Maturity  Model  (CMM)  Workshop  and  a  United  States  Department  of 
Defense  (DoD)  Software  Process  Workshop,  which  were  held  in  parallel  on  April  5  and  6.  A 
related  series  of  tutorials  were  held  on  April  10.  A  number  of  the  participants  attended  one  or 
more  of  the  series  of  events.  During  the  workshops  the  participants  engaged  in  interactive 
panel  discussions  and  working  sessions,  listened  to  keynote  and  lunchtime  speakers,  and 
discussed  topics  of  mutual  interest  at  ‘birds  of  a  feather”  sessions.  The  SEI  provided 
participants  with  a  notebook  containing  materials  pertaining  to  the  workshop(s)  they  attended. 

This  report  is  based  on  the  SEPG  Workshop:  Questionnaire  &  Event  Evaluation  that  was 
distributed  on  April  9,  the  last  day  of  the  SEPG  Workshop.  Participants  were  encouraged  to 
jomplete  the  questionnaires  and  return  them  as  they  left  one  of  the  mid-aftemoon  general 
sessions.  In  all,  379  people  attended  the  SEPG  Workshop.  We  distributed  questionnaires  to 
240  participants  who  remained  at  the  workshop  on  the  afternoon  of  April  9.  Of  the  240  people 
who  received  the  questionnaires,  169  completed  and  returned  them,  representing 
approximately  70  percent  of  those  who  received  the  questionnaires.^ 


'  The  participants  who  completed  the  survey  constitute  a  good  sample  of  those  in  attendance  at  the  workshop,  at 
least  on  its  last  day.  However  those  who  attended  the  workshop  are  self  selected,  and  are  not  necessarily 
representative  of  all  software  process  improvement  efforts. 
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This  report  contains  six  chapters  following  this  introduction.  Chapter  2  summarizes  the 
participants’  judgements  about  the  quality  of  the  SEPG  Workshop  and,  for  comparative 
purposes,  also  contains  some  limited  information  about  their  ratings  of  the  CMM  Workshop. 
A  description  of  the  SEPGs  currently  functioning  in  the  participants’  home  organizations  is 
contained  in  Chapter  3,  while  the  software  process  improvement  efforts  of  those  organizations 
are  described  in  Chapter  4.  Chapters  5  and  6  provide  background  information  about  the 
software  development  work  done  in  the  participants’  organizations  and  about  the  participants 
themselves.  A  summary  of  the  report  findings  is  contained  in  Chapter  7.  A  copy  of  the 
questionnaire  can  be  found  in  the  appendix  to  this  report. 
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2  Evaluation  of  the  SEPG  Workshop 


This  section  provides  a  brief  summary  evaluation  of  the  SEPG  Workshop  from  the  perspective 
of  the  participants  who  completed  the  SEPG  Workshop:  Questionnaire  &  Event  Evaiuation. 
Their  evaluative  judgments  about  the  quality  of  the  SEPG  Workshop  are  based  on  both  fixed 
response  “closed-ended”  and  free  format  “open-ended"  questions. 

2.1  Ratings  for  the  Workshop 

The  participants  rated  the  overall  workshop  and  six  of  its  areas  on  a  five  point  scale  ranging 
from  "inadequate"  to  "excellent.”  The  areas  reviewed  include  the: 

•  Workshop  in  general 

•  Keynote  speakers 

•  Panels 

•  Working  sessions 

•  Workshop  notebook 

•  “Birds  of  a  feather”  sessions 

The  workshop  in  general  received  very  favorable  ratings.  As  shown  in  Figure  1 , 54  percent  of 
the  participants  rated  the  workshop  in  general  on  the  "excellent”  side  of  "OK.”  Only  15  percent 
gave  it  ratings  on  the  “inadequate”  side  of  "OK.” 


Figure  1.  Evaluation  of  the  Workshop  in  General  (n=160) 
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Of  the  six  areas  reviewed  by  the  participants,  the  keynote  speakers  received  the  most  positive 
ratings.  In  all,  79  percent  of  those  who  replied  rated  the  keynote  speakers  favorably  while  only 
1  percent  of  those  who  replied  found  the  keynote  speakers  to  be  inadequate.  These  results 
are  displayed  in  Figure  2. 


Inadequate  OK  Excellent 


Figure  2.  Evaluation  of  the  Keynote  Speakers  (n=l65) 


The  panels  also  received  favorable  reviews.  However,  the  ratings  for  the  panels  center  more 
closely  around  the  "OK"  choice.  As  shown  in  Figure  3, 39  percent  of  the  participants  gave  the 
panels  a  favorable  rating.  Yet  very  few  participants  chose  the  more  definite  “excellent” 
category.  Both  the  "inadequate"  and  "excellent"  ratings  received  only  about  five  percent  of  the 
responses.^ 


60 


Figure  3.  Evaluation  of  the  Panels  (n=l64) 


^  The  participants  typically  attended  several  panels,  so  it  is  likely  that  the  responses  are  grouped  towards  the 
middle  because  people  had  mixed  reactions,  both  positive  and  negative,  to  the  different  panels. 


4 


CMU/SEI-92-SR-13 


The  working  sessions  received  less  favorable  ratings  than  did  the  keynote  speakers  and  panel 
sessions,  receiving  a  combined  favorable  rating  of  34  percent.  As  shown  in  Figure  4,  a  fulMO 
percent  of  those  responding  found  the  working  sessions  to  be  inadequate. 


60 

50 


Figure  4.  Evaluation  of  the  Woildng  Sessions  (nsl63) 


As  shown  in  Figure  5,  the  workshop  notebook  also  received  less  favorable  reviews  than  the 
other  areas  reviewed.  Of  those  answering,  32  percent  gave  the  notebook  a  favorable  rating, 
while  34  percent  found  the  notebook  to  be  less  than  “OK.”  In  response  to  the  free  format, 
open-ended  responses  summarized  in  Tables  2  and  6,  participants  stated  the  notebook  was 
“incomplete,”  “disorganized  and  confusing,”  and  “far  too  large,  save  a  tree.”  The  notebook  is 
a  major  reference  document  representing  the  SEI,  which  the  participants  will  return  to  their 
organizations.  Therefore,  the  quality  of  the  notebook  needs  to  be  addressed  for  future  SEPG 
Workshops. 


60 


50 


Figures.  Evaluation  of  the  Workshop  Notebook  (nsiss) 
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Twenty-seven  percent  of  the  respondents  gave  the  “birds  of  a  feather"  sessions  a  favorable 
rating.  An  equal  number  of  participants  (six  percent)  chose  each  of  the  two  extreme 
responses.  In  fact,  the  participants  are  almost  equally  distributed  across  the  rating  scale,  with 
44  percent  of  the  participants  indicating  that  the  “birds  of  a  feather”  sessions  were  “OK.”  The 
results  are  displayed  in  Figure  6.  Notice  that  fewer  participants  attended  these  sessions  than 
attended  the  other  workshop  events.^  This  observation  may  be  explained  by  the  fact  that  the 
“birds  of  a  feather”  sessions  were  held  in  the  evening.  As  seen  in  Table  3  on  page  8,  many  of 
the  participants  stated  that  the  days  were  too  long.  It  is  also  worth  noting  that  the  “birds  of  a 
feather”  sessions  were  not  fully  advertised  in  the  workshop  materials. 


60' 

50' 

40' 

I  30' 

20' 

10' 

O' 

Inadequate  OK  Excellent 

Figure  6.  Evaluation  of  the  “Birds  of  a  Feather”  Sessions  (n=70) 


2.2  Descriptions  of  the  Workshop 

The  participants  were  asked  to  describe  in  their  own  words  what  they  liked  most  and  least 
about  the  workshop.  The  questions  were  open-ended,  so  people  could  list  several  different 
areas  if  they  so  chose.  Table  1  summarizes  their  responses  about  what  they  liked  most.  Two 
areas  clearly  stood  out  in  the  minds  of  the  participants:  the  "birds  of  a  feather”  sessions  and 
the  keynote  speakers.  Over  20  percent  of  the  SEPG  Workshop  attendees  mentioned  each  of 
these  two  areas  as  aspects  of  the  workshop  that  they  liked  most. 

It  is  interesting  to  note  that  the  working  sessions  also  received  a  significant  number  of 
responses  with  12  percent  indicating  in  their  responses  that  they  preferred  these  sessions  the 
most.  Contrary  to  the  impression  that  the  comparable  closed-ended  question  may  give  about 
the  working  sessions  in  general,  at  least  some  of  the  working  sessions  were  particularly  well 


^  How  many  sessions  the  participants  attended  is  unclear  from  these  data  alone  because  different  participants 
may  be  rating  different  sessions,  rather  than  rating  the  same  sessions  as  being  uniformly  middle  ground. 
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received  by  some  of  the  attendees.^  Indeed,  one  of  the  strongest  indicators  of  whether  a 
component  of  the  workshop  was  successful  is  whether  or  not  the  respondents  believe  it  should 
be  a  part  of  future  SEPG  Workshops.  As  seen  in  Table  2.  almost  two-thirds  of  the  respondents 
believe  that  working  sessions  should  remain  a  part  of  future  SEPG  Workshops. 


Table  1 .  Areas  That  Participants  Liked  Most  About  the  Workshop'*  (nsl69) 


Keynote  speakers 

23 

“Birds  of  a  feather”  sessions 

21 

Informal  working  sessions 

12 

Meals 

11 

Exposure  to  broad  set  of  experience 

9 

Helpful  information 

9 

Interaction  with  people  with  common  interest 

9 

Networking  opportunities 

7 

Its  organization 

7 

Did  not  respond 

12 

^  Based  on  responses  to  the  question:  “What  did  you  IHte  the  most  about  the  workshop?’ 

^  Multiple  responses  were  possible.  The  table  excludes  entries  for  categories  mentioned  by  fewer  than  ten 
workshop  participants. 

Table  2.  Should  Working  Sessions  Continue  to  Be  Included?'*  (nr  160) 
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the  participants  pointed  out  that  information  was  missing  from  the  notebook.  This  information 
coupied  with  the  ratings  reported  in  Figure  6  indicate  a  problem  that  should  be  addressed  in 


future  workshops. 

Table  3.  Areas  That  Participants  Liked  Least  About  the  Workshop''  (nsl69) 


Percent  responding^ 

Too  long 

31 

Materials  missing  from  notebook 

16 

Too  crowded 

13 

Lack  of  substance  in  many  talks 

10 

Lack  of  experience  of  presenters 

8 

Night  sessions 

7 

Lack  of  organization 

7 

Did  not  respond 

14 

'  Based  on  responses  to  the  question:  *What  did  you  like  the  least  about  the  workshop?’ 

^  Multiple  responses  were  possible.  The  table  excludes  entries  for  categories  mentioned  by  fewer  than  ten 
workshop  participants. 

To  gauge  the  overall  quality  of  the  event,  the  participants  were  also  asked  if  they  would  attend 
an  SEPG  Workshop  in  the  future.  The  assumption  here  Is  that  people  will  willingly  choose  to 
repeat  a  good  experience.  As  seen  in  Table  4,  almost  three-quarters  of  the  participants  said 
that  they  would  choose  to  attend  an  SEPG  Workshop  in  the  future.  More  significantly,  only 
three  percent  indicated  that  they  would  not  attend  another  SEPG  Workshop.  Those 
participants  expressing  a  preference  to  attend  another  SEPG  Workshop  also  indicated  that  it 
should  be  held  annually,  with  79  percent  of  them  saying  that  the  next  workshop  should  be  held 
in  1993. 


Table  4.  Participants’ Willingness  to  Attend  Future  SEPG  Workshops''  (n=169) 


Percent  responding^ 

Yes 

73% 

No 

3 

Don't  know 

21 

Did  not  respond 

_2 

99% 

^  Based  on  responses  to  the  question:  Would  you  attend  another  SEPG  workshop  in  the  future?’ 

^  The  table  does  not  total  to  1 00%  due  to  rounding  error. 

Participants  were  asked  the  open-ended  question  'What  would  you  change  for  future 
workshops?”  Once  again  some  participants  mentioned  more  than  one  thing,  so  the  answers 
are  categorized  allowing  for  multiple  responses.  As  can  be  seen  in  Table  5,  there  were 
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several  areas  that  the  participants  felt  required  improvement  in  the  future.  The  two  most 
common  sets  of  suggestions  were  to  shorten  the  days  and  increase  the  substance/expertise 
of  the  presentations. 


Tables.  Suggested  Changes  for  Future  Workshops''  (nsl3l) 


Percent  responding^ 

Limit  work  hours 

12 

Focus  on  a  few  key  areas 

9 

More  expert  preparation 

9 

More  substance 

8 

Present  more  personal  experience 

7 

More  expert  lecturers 

6 

Provide  materials  needed 

6 

Did  not  respond 

22 

'  Based  on  responses  to  the  question  ‘What  would  you  change  for  future  workshops?' 

^  MuKiple  responses  were  possible.  The  table  excludes  entries  for  categories  mentioned  by  fewer  than  ten 
workshop  participants. 

The  final  question  on  the  event  evaluation  asked  ttie  participants  their  primary  reasons  for 
attending  the  SEPG  Workshop.  The  most  common  single  answer  categorized  in  Table  6 
refers  to  interaction  with  participants  from  other  companies.  Not  surprisingly,  answers  also 
commonly  referred  to  gaining  knowledge  about  SEPGs  and  the  CMM. 


Tables.  Primary  Reason  for  Attending  SEPG  Workshop''  (nsl69) 


Percent  responding^ 

Get  help  and  ideas  from  other  companies 

21 

Gather  information  on  other  SEPGs 

15 

Gain  general  knowledge 

12 

Learn  more  about  CMM  intent 

12 

Learn  more  about  SEPGs 

11 

Learn  more  about  software  process  improvement 

11 

Learn  how  to  start  an  SEPG 

7 

Did  not  respond 

8 

1  Based  on  responses  to  the  question  ‘What  was  your  primary  reason  for  attending  the  SEPG  workshop?” 

^  Multiple  responses  were  possible.  The  table  excludes  entries  for  categories  mentioned  by  fewer  than  ten 
workshop  participants. 
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2.3  Evaluation  of  the  CMM  Workshop 

The  event  evaluation  for  the  SEPG  Workshop  was  our  major  concern,  but  we  also  included 
two  specific  questions  about  the  CMM  Workshop.  The  first  question  was  included  in  the  series 
asking  for  ratings  about  aspects  of  the  week  long  event  and  asked  the  participants  to  rate  the 
CMM  disposition  reviews.  The  second  question  asked  those  who  said  that  they  would  attend 
another  SEPG  Workshop  whether  it  should  be  held  in  conjunction  with  another  CMM 
Workshop. 

The  CMM  Workshop  disposition  reviews  received  favorable  ratings,  although  they  were  not  as 
strong  as  the  general  ratings  for  the  SEPG  Workshop.  Fully  half  of  those  who  replied  rated 
the  CMM  Workshop  disposition  reviews  towards  the  more  favorable  end  of  the  continuum  of 
possible  responses.^  As  shown  in  Figure  7. 16  percent  rated  the  CMM  Workshop  disposition 
reviews  as  "excellent."  However,  25  percent  rated  the  reviews  as  being  less  than  "OK,”  with 
8  percent  rating  them  as  being  "inadequate.” 


60 

50 


Inadequate  OK  Excellent 


Figure?.  Evaluation  of  the  CMM  Workshop  (n=88) 


^  Fewer  people  answered  this  question  since  not  all  of  those  who  attended  the  SEPG  Workshop  had  also  attended 
the  CMM  Workshop  held  earlier  in  the  week. 
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Finally,  the  idea  of  combining  the  CMM  and  SEPG  Workshops  in  a  common  event  did  receive 
support  from  those  SEPG  Workshop  participants  who  expressed  an  interest  in  attending  a 
future  SEPG  event.  As  seen  in  Table  7,  over  60  percent  indicated  that  the  CMM  Workshop 
should  be  held  in  conjunction  with  a  future  SEPG  Workshop.  While  over  a  quarter  of  those 
responding  were  not  sure,  13  percent  said  that  the  two  events  should  not  be  held  together. 


Table  7.  Should  SEPG  &  CMM  Workshops  Be  Held  In  Conjunction?^  (nsi33) 


Yes 

Percent  responding 

61% 

No 

13 

Don't  know 

2& 

100% 

'  Based  on  responses  to  the  question  ‘Should  it  be  held  in  conjunction  with  another  CMM  Workshop?’ 
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3  Status  of  the  SEPGs 


The  1992  SEPG  Workshop  provided  a  unique  opportunity  to  gather  information  about  the 
nature  of  software  engineering  process  groups  and  software  process  improvement  efforts  as 
they  have  been  implemented  thus  far.  The  workshop  participants  are  a  self  selected  group 
and  do  not  constitute  a  randomly  selected  sample  of  software  development  organizations. 
Nevertheless,  the  questionnaire  responses  can  provide  some  useful  insight,  at  least  from  the 
perspective  of  organizations  who  base  their  work  on  the  approach  of  the  SEI  and  CMM. 

3.1  Characteristics 

The  first  series  of  questions  we  asked  about  the  SEPGs  focused  on  their  organization.  Not 
surprisingly,  almost  three-quarters  (121  of  169)  of  the  workshop  participants  who  completed 
a  questionnaire  reported  that  their  organizations  have  an  SEPG.  Of  those  who  have  an 
SEPG,  48  percent  indicated  that  it  exists  at  the  organization  level,  while  43  percent  specified 
the  division  level. 

Who  in  an  organization  demonstrates  commitment  to  an  SEPG  can  be  crucial  to  the  success 
of  a  software  process  improvement  effort.  Table  8  shows  where  commitment  to  the 
participants’  SEPGs  has  been  demonstrated.  Note  that  only  about  a  third  (37  percent)  of  the 
participants  indicated  that  their  technical  staffs  had  demonstrated  a  commitment  to  their 
SEPGs.  This  can  be  compared  to  three-quarters  (67  percent)  who  indicated  support  from 
senior  management,  and  41  percent  who  checked  the  middle  management  category.  This 
mix  of  commitment  may  indicate  a  potential  hurdle  that  an  SEPG  must  overcome  once  it  has 
received  management  approval. 


Tables.  Organizational  Commitment  to  SEPGs''  (n=l2l) 


Senior  management 

Middle  management 

41 

Technical  staff 

37 

^  Based  on  responses  to  the  question;  “Who  in  the  organization  has  demonstrated  their  commitment  to  your 

SEPG?’ 

^  Column  totals  to  more  than  100%  due  to  multiple  responses. 

Most  of  these  SEPGs  were  established  after  the  SEI  Technical  Report  SEI-TR-23  was  issued 
in  1987.®  Only  seven  participants  report  groups  that  predate  this  report.  The  earliest  starting 
date  is  1961,  with  a  17-year  hiatus  before  the  second  one.  Seventy-eight  percent  were 
established  since  1990,  with  forty  percent  being  established  in  1991  alone.  Clearly  these 
SEPGs  are  still  relatively  new. 

®  Humphrey,  W.S.  and  Sweet,  W.L,  4  Method  for  Assessing  the  Software  Engineering  Capability  of  Contractors, 
CMU/SEI-87-TR-23,  ADA1 87320,  Software  Engineering  Institute,  September  1987. 
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The  participants’  SEPGs  vary  in  size  as  well.  To  gauge  typical  staff  size,  the  participants  were 
asked  to  record  the  number  of  people  who  work  in  their  SEPGs.  As  seen  in  Figure  8,  the 
m^ority  of  the  SEPGs  described  have  eight  or  fewer  full-time  and/or  part-time  staff  members. 
Half  of  the  SEPGs  have  a  maximum  of  three  full-time  and/or  four  part-time  staff  members.  A 
small  number  of  SEPGs  (three)  have  25  or  more  full-  or  part-time  staff  members. 


Based  on  responses  to  the  question:  “How  many 


0  1  2  3  4  5  6  7  8  9  10  11  12  14  15  20  25  30  40  45  50 


Staff  size 


_  Full-time 

Part-time 

Consulting 

(n=115) 

(n=96) 

(n=53) 

Figures.  SEPG  Staff  Size 


3.2  Activities 

The  workshop  participants  who  indicated  that  their  organizations  had  SEPGs  were  asked  four 
free  response  “open-ended”  questions  about  their  SEPGs.  Similar  responses  were  classified 
as  described  below.  These  answers  provide  needed  insight  into  the  history  and  activities  of 
SEPGs. 

The  first  question  asked  the  participants  to  describe  the  functions  performed  by  their  SEPGs. 
Notice  in  Table  9  that  references  to  “coordination"  activities  are  most  common.  These  include 
general  comments  about  intergroup  coordination,  as  well  as  mentions  of  newsletters,  related 
communications  activities,  consulting,  and  change  management.  The  next  most  common 
category  includes  references  to  “process  improvement,”  which  also  tend  to  be  general  in 
nature.  However,  some  of  these  comments  do  mention  responsibilities  for  implementing 
specific  recommendations  based  on  software  process  assessments. 
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Tables.  SEPG Functions'*  (n=l2l) 


_ 


Coordination 

60 

Process  imp'’ovement 

45 

Software  support  function(s) 

44 

Training  /  Education 

29 

Facilities^ 

19 

Metrics 

15 

Software  quality  assurance 

3 

Process  definition 

37 

Assessments  /  Evaluations 

29 

Action  plan  execution 

17 

Standards 

15 

Improvement  of  CMM  Level 

4 

Other^ 

7 

Don’t  know 

1 

Does  not  apply  -  SEPG  too  new 

4 

Did  not  respond 

7 

'  ‘Based  on  responses  to  the  question:  “What  functions  does  your  SEPG  perform?" 

^  Column  totals  to  more  than  100%  due  to  multiple  responses. 

^  Includes  oonfiguration  management,  software  engineering  environments  (SEE),  CASE,  and  reuse. 

*  Includes  references  to  writing  and  reviewing  proposals  (2),  reviewing  requirements  (2),  staffing  software 

related  inKiatives,  project  planning  and  requirements  management,  collecting  lessons  learned,  and  “Not 

much  at  the  current  time.* 

1 

Not  surprisingly  many  SEPGs  (44  percent)  perform  traditional  software  support  functions,  with 
responsibility  for  formal  training  programs  being  most  common.  As  seen  in  Table  9,  SEPG 
functions  also  commonly  include  CMM  based  process  definition  activities,  and  conduct  of 
process  assessments  or  participation  in  capability  evaluations. 

The  second  open-ended  question  asked  the  participants  to  identify  their  SEPGs’  primary 
responsibilities.  As  shown  in  Table  1 0.  their  answers  differ  from  those  in  response  to  the  more 
general  question  about  SEPG  functions.  When  answering  the  general  question,  people  often 
enumerate  several  functions.  However,  they  typically  mention  fewer  activities  as  primary 
responsibilities  for  their  SEPGs.  Thus,  other  things  being  equal,  the  individual  percentage 
entries  should  be  expected  to  be  lower  in  Table  10  than  in  Table  9. 

The  response  patterns  in  the  two  tables  differ  in  other  ways  as  well.  Mentions  of  process 
improvement  overtake  coordination  as  the  most  common  category  of  SEPG  primary 
responsibilities.  In  fact,  more  people  (53  percent)  mentioned  process  improvement  as  a 
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primary  responsibility  than  as  an  SEPG  function  in  general  (45  percent).  While  many  (44 
percent)  of  the  SEPGs  do  perform  traditional  software  support  functions,  considerably  fewer 
(1 1  percent)  claim  such  functions  as  their  primary  responsibilities. 


Table  10.  SEPG  Primary  Responsibilities''  (n=l2l) 


Percent  performing  function® 

Process  improvement 

53 

Coordination 

42 

Process  definition 

19 

Improvement  of  CMM  level 

14 

Support  function(s) 

11 

Training  /  Education 

2 

Facilities® 

8 

Metrics 

5 

Action  plan  execution 

11 

Standards 

10 

Assessments  /  Evaluations 

5 

Other^ 

4 

Don’t  know 

1 

Does  not  apply  -  SEPG  too  new 

2 

Did  not  respond 

9 

'  Based  on  responses  to  the  question:  ‘What  is  your  SEPG’s  primary  responsibility?” 

^  Column  totals  to  more  than  1 00%  due  to  multiple  responses. 

^  Includes  configuration  management,  software  engineering  environments  (SEE),  CASE,  and  reuse. 

*  Includes  references  to  collecting  ‘lessons  learned,”  improving  software  product  quality  and  time  to  market. 

”general  question  ->  general  answer,”  ‘Not  much  at  the  current  time.. 

.,”  and  “Good  Question. 

■ 

Finally,  several  people  (14  percent)  report  that  improving  their  organizations’  maturity  levels  is 
a  primary  SEPG  responsibility  mandated  by  their  management.  Such  references  do  not 
necessarily  indicate  an  uncritical  attempt  to  reach  “level  2  in  ’92"  (or  level  3  in  ’93).  Indeed 
some  of  them  may  be  shorthand  among  those  familiar  with  the  CMM  for  a  commitment  to  real 
process  improvement.  However,  such  responses  do  raise  a  legitimate  concern  about  the 
degree  of  sophistication  in  the  community  about  the  CMM  and  its  application. 

3.3  Initiation 

A  third  open-ended  question  asked  the  respondents  to  describe  the  planning  and  related 
activities  that  it  took  to  start  their  SEPGs.  Their  responses  are  summarized  in  Table  1 1 .  Well 
over  a  third  of  those  responding  highlighted  the  importance  of  support  from  managers  who  can 
be  made  to  appreciate  the  need  for  process  improvement.  Of  course,  those  who  would  like 
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to  see  viable  SEPGs  established  cannot  necessarily  choose  their  own  enlightened 
organizational  management.  However,  the  references  to  the  importance  of  individual 
champions  (23  percent)  and  steering  committees  and  related  working  groups  (21  percent) 
indicate  that  the  workshop  participants  feel  that  hard  work  can  pay  off.  In  a  related  vein,  close 
to  30  percent  of  these  individuals  stressed  the  importance  of  having  done  a  software  process 
assessment  as  a  precursor  to  establishing  their  SEPGs.^ 


Table  1 1 .  Activities  Required  to  Start  SEPGs^  (n=l  21 ) 


Management  sign-on 

Percent  responding 

37 

Senior  management 

29 

Middle  management 

16 

Conducted  assessments  and/or  evaluations 

27 

A  dedicated  champion 

23 

Group  -  Committee  work 

21 

Secured  needed  resources 

8 

Threat  of  SCE  directives 

6 

Help  from  the  SEI 

6 

Enhancement  to  existing  organization 

4 

Sign-on  of  technical  staff 

3 

Other^ 

4 

Don’t  know 

11 

Does  not  apply  -  SEPG  too  new 

2 

Did  not  respond 

12 

^Based  on  responses  to  the  question:  “What  kind  of  planning  and  related  activities  did  it  take  to  get  your  SEPG 
started?’ 

^  Column  totals  to  more  than  1 00%  due  to  multiple  responses. 

®  Responses  include  references  to  ’reoognition...[of)  need...,’  “Stupid  question.  Not  near  enough  room  or 
time  to  discuss  here,’  “Identify  needs...,’  “Ad-hoc  start  (hallway  discussion),’  and  help  from  a  consultant. 


3.4  Accomplishments 

The  final  open-ended  question  in  this  series  asked  about  the  major  accomplishments 
attributable  thus  far  to  the  participants’  SEPGs.  Their  answers  are  summarized  in  Table  12. 
Notice  there  that  a  large  majority  described  planning  oriented  activities.  Those  classified  as 
"CMM  based”  include  references  to  development  of  new  action  plans,  process  definitions,  and 


’’  Most  of  these  references  were  explicitly  to  SEI  assessments  or  self-assessments.  A  few  references  were  made 
to  capability  evaluations  (SCEs);  however  they  mentioned  positive  experiences  rather  than  the  compulsion  of 
having  to  do  the  SCEs. 
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important  software  development  standards.  The  "communication  and  education”  category 
includes  references  to  activities  meant  to  improve  organizational  commitment  to  process 
improvement.® 

Table  12.  Major  SEPG  Accomplishments''  (nsi2l) 


Percent  responding 

Process  planning 

70 

CMM  based  activities 

57 

Communication  -  Education 

35 

Secured  staff  -  Management  commitment 

9 

Process  implementation 

59 

Action  plans 

36 

New  assessments 

25 

Metrics  programs 

12 

Organizational  infrastructure 

9 

Methods  and  tools® 

9 

Testing  program 

1 

SQA 

1 

Other^ 

4 

Don’t  know 

1 

Does  not  apply  -  SEPG  too  new 

7 

Did  not  respond 

8 

'  Based  on  responses  to  the  question:  'What  are  the  major  accomplishments  of  your  SEPG  thus  far?” 

^  Column  totals  to  more  than  100%  due  to  multiple  responses. 

^  Includes  references  to  the  establishment  of  configuration  management,  SEE,  CASE,  and  reuse  facilities. 
*  includes  references  to  “survivar  (2)  and  “nothing  as  yet’  (2). 


However,  major  SEPG  accomplishments  described  by  the  respondents  are  not  limited  to 
planning  efforts.  Notice  also  in  Table  1 2  that  close  to  60  percent  of  the  respondents  described 
at  least  some  changes  in  their  organizations’  normal  way  of  doing  business.  Participants  had 
to  mention  tangible  changes  for  a  response  to  be  classified  as  “process  implementation.” 
Planning  alone  was  not  enough.  For  example,  participants  had  to  claim  actual  plan 
implementation  to  be  included  in  the  "action  plans”  category.  Similarly,  the  "organizational 
infrastructure”  category  includes  references  to  the  establishment  of  organizational  entities  and 
resources  meant  to  maintain  long-term  process  improvement  activities. 


These  include  a  few  specific  references  to  newly  instituted  training  programs,  which  might  better  be  classified  as 
actual  implementation  rather  than  planning  only. 
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4  Status  of  Process  Improvement  Efforts 
4.1  Improvement  Plans 

The  participants  were  asked  if  their  organizations  have  developed  software  process 
improvement  plans.  As  seen  in  Table  13, 63  percent  stated  that  they  did  have  some  type  of 
software  process  improvement  plans. 


Table  13.  Existence  of  a  Software  Process  Improvement  Plan''  (nsi69) 


Percent  resoondinp^ 

Yes 

63% 

For  the  organization 

35 

For  my  division 

26 

Elsewhere 

15 

No 

34 

Did  not  respond 

Ji 

100% 

'  Based  on  responses  to  the  question;  ‘Has  your  organization  developed  a  software  process  improvement 
plan?* 

^  The  ‘yes*  sub-column  totals  to  more  than  63%  due  to  multiple  responses. 

Those  who  reported  that  their  organizations  have  a  software  process  improvement  plan  were 
asked  to  describe  the  method(s)  on  which  the  plans  are  based.  The  responses  are 
summarized  in  Table  1 4.  The  large  majority  (81  percent)  reported  that  their  plans  were  based 
on  SEI  methods,  specifically  either  the  CMM  and/or  TR-23.  Over  60  percent  reported  reliance 
on  TR-23  while  over  one  half  indicated  that  they  were  currently  basing  their  work  at  least  partly 
on  the  CMM.  Moreover,  much  of  the  "other  process  improvement  work”  was  based  on  SEI 
methods. 


Table  14.  Basis  for  Software  Process  Improvement  Plan  (n=l07) 


Percent  responding 

SEI  Methods 

81 

TR-23 

61 

The  CMM 

54 

Other  process  improvement  work 

48 

SEi-styie  self  assessments 

18 

SEI  &  non-SEI  mixed  methods 

7 

Other  methods^ 

26 

^  Both  columns  total  to  more  than  1 00%  due  to  multiple  responses. 

2  Includes  references  to  general  knowledge  (11),  TQM  (3).  Deming  (4),  Joiner  (2).  Crosby  (2).  Jones  (1).  Juran 
(1),  Radice  (1),  2167a  and  2168  (1).  Two  respondents  expressed  confusion,  while  several  did  not  clarify 
their  choice. 
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4.2  Obstacles  to  and  Facilitators  of  Process  Improvement 

All  of  the  respondents  to  the  questionnaire  were  asked  to  describe  any  major  obstacles  to 
software  process  improvement  in  their  organizations,  regardless  of  whether  or  not  they  have 
an  SEPG.  A  companion  question  asked  what  facilitated  their  process  improvement  efforts. 

4.2.1  Obstacles 

Notice  in  Table  15  that  management  is  the  most  commonly  mentioned  obstacle  to  software 
process  improvement.  People's  comments  include  references  to  management  resistance, 
lack  of  support  from  management,  and  misunderstanding  of  the  importance  of  process 
improvement.  Consider,  for  example,  the  following  response. 

Management  does  not  want  the  general  guidelines  we  have  given  them.  They 
want  the  Cliff  Notes.  They  do  not  want  guidelines  and  assessments.  They  want 
processes  and  rules. 

Given  the  number  of  people  who  mention  lack  of  support  from  management,  it  is  not  surprising 
that  shortages  of  staff  and  other  SEPG  resources  are  the  next  most  commonly  mentioned 
obstacle  to  process  improvement.  These  are  followed  by  references  to  inadequacies  of  the 
current  organizational  structure  and  “culture.” 

The  “organizational  culture"  category  includes  vague  references  to  cultural  resistance  and 
skepticism  about  process  improvement  and  total  quality  management  (TQM).  On  the  other 
hand,  responses  classified  as  “organizational  structure”  refer  to  specific  problems  in  the 
current  organization  that  inhibit  change.  It  may  be  difficult  for  supporters  of  process 
improvement  to  do  much  about  corporate  culture  as  evidenced  by  the  following  comment. 

[Our]  culture  is  not  data-oriented.  [The]  organization/culture  is  proud  of  being 
non-DoD  and  non-standardized. 

However,  structural  references  to  "imposed  schedules  from  marketing”  or  “differences  in 
approaches  in  software  development  and  support”  are  more  tangible  and  possibly  amenable 
to  change. 

Some  of  the  “culture”  comments  clearly  are  indicative  of  substantial  impediments  to  change. 
For  example,  a  "hardware  based  culture”  will  not  change  quickly.  However,  even  some  of  the 
“culture”  comments  are  tangible  enough  to  provide  direction  for  change  (for  example, 
“Establishing  a  culture  of  analysis  and  planning  prior  to  commitment"). 

While  references  to  management  commitment  are  much  more  common,  notice  in  Table  15 
that  quite  a  few  respondents  (12  percent)  also  mention  the  importance  of  gaining  commitment 
from  the  engineering  and  technical  staff.  Recall  from  Table  1 1  that  support  from  the  technical 
staff  was  mentioned  relatively  infrequently  (three  percent)  as  being  important  in  establishing 
SEPGs.  Unlike  management  support,  widespread  technical  support  does  not  appear  to  be  a 
necessary  condition  for  establishing  an  SEPG  as  an  organizational  entity.  However,  a  lack  of 
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active  support  from  technical  people  is  more  likely  to  be  mentioned  as  an  obstacle  to  the 
success  of  a  broader  software  process  improvement  effort. 

Although  the  percentage  is  relatively  low  (six  percent),  it  is  also  worth  emphasizing  that  some 
individuals  discussed  the  lack  of  evidence  about  return  on  investment  as  a  major  obstacle  to 
software  process  improvement.  Many  or  most  of  the  people  who  attend  a  function  like  the 
SEPG  Workshop  are  themselves  likely  to  believe  that  a  software  process  improvement  effort 
dearly  is  a  worthwhile  endeavor.  It  is  noteworthy  that  even  some  of  these  “believers" 
mentioned  return  on  investment  as  an  obstacle. 

Tablets.  Major  Obstacles  to  Software  Process  Improvement  (n=157)^ 


Percent  responding^ 

Management  43 

Inadequate  SEPG  resources  22 

Organizational  structure  17 

Organizational  culture  13 

Engineering  /  Technical  staff  1 2 

No  Evidence  of  return  on  investment  6 

Too  early  to  tell  4 

Other®  7 

Don't  know  1 

Did  not  respond  22 

Does  have  an  SEPG  1 8 

Also  did  not  answer  companion  (fadlitation)  question  5 

Did  answer  companion  (fadlitation)  question  1 7 


^  Based  on  responses  to  the  question:  'What  major  obstacles  must  a  software  process  improvement  effort 
overcome  in  your  organization  and/or  division?' 

^  There  are  people  who  attenoed  the  workshops  and  filled  out  the  questionnaire  who  are  consultants  or  who 
come  from  organizations  that  do  not  themselves  do  software  development  work.  Twelve  individuals 
indicated  in  response  to  either  this  question  and/br  the  one  about  facilitators  that  the  question  does  not  apply 
to  them  for  those  reasons,  and  they  are  excluded  from  the  figures  reported. 

^  Column  totals  to  more  than  1 00%  due  to  multiple  responses. 

*  Senior  and  middle  management  references  are  combined  because  it  is  difficult  to  distinguish  between  the 
two  in  many  of  these  responses.  Senior  management,  however,  seems  to  be  implied  more  often. 

^  Responses  include  reference  to  an  organizational  emphasis  on  reaching  'an  SEI  maturity  level  by  a  given 
date,*  the  failure  to  have  done  a  process  assessment,  the  need  for  a  'usable  process  definition,*the  'inertia 
of  existing  methods,*  the  idea  that  *SQA  will  not  be  separate  in  the  Air  Force,*  the  need  for  'howto*  education 
for  action  planning,  the  existence  of  past  failures,  the  need  to  demonstrate  successes  (2),  and  comments 
that  we  were  unable  to  interpret  (3). 


The  number  of  people  who  failed  to  answer  this  question  is  considerably  higher  than  the 
number  of  people  who  did  not  answer  the  questions  asking  about  the  SEPGs.  However, 
notice  in  the  table  that  a  large  proportion  of  those  who  failed  to  answer  the  question  about 
obstacles  did  answer  the  companion  question  about  facilitators.  Only  five  percent  answered 
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neither  question.  Simiiarly.  a  large  proportion  of  those  who  failed  to  answer  are  also  those 
who  report  having  SEPGs  in  their  organizations.  At  least  some  of  these  people  may  have 
failed  to  answer  the  question  because  they  could  not  think  of  any  particular  obstacles  to 
software  process  improvement. 

4.2.2  Facilitators 

As  mentioned  earlier,  we  also  asked  people  to  identify  what  in  particular  has  facilitated  their 
process  improvement  efforts.  Once  again,  the  most  common  answers  emphasized  the 
importance  of  management  “signing  on"  and/or  making  top-level  command  decisions.  Notice 
in  Table  16  that  responses  classified  as  “demand”  are  the  next  most  common  facilitators. 
These  responses  include  references  to  SCE  directives  and  other  customer  demands  for 
process  improvement.  Answers  classified  as  referring  to  “organizational  incentives”  include  a 
desire  to  enhance  corporate  market  share,  the  existence  of  other  quality  initiatives  such  as 
TOM,  and  mention  of  executive  bonuses. 

Since  the  CMM  emphasizes  management,  it  is  not  surprising  that  more  references  are  made 
to  the  importance  of  management  than  of  technical  staff.  However,  a  substantial  minority  (15 
percent)  did  respond  to  this  general,  open-ended  question  by  discussing  the  importance  of  the 
technical  staff  in  facilitating  process  improvement 
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Table  16.  Major  Facilitators  of  Software  Process  improvement^  (nslST)^ 


Management 

Per 

cent  responding^ 
48 

Senior  management 

35 

Middle  management 

23 

Demand 

22 

Organizational  incentives 

15 

Engineering  /  Technical  staff 

15 

Dedicated  champion 

11 

Help  from  the  SEI 

9 

SEPG  efforts 

8 

Assessments  /  Evaluations 

5 

Group  -  Committee  work 

4 

Secured  needed  resources 

4 

Consultants 

2 

Other^ 

6 

Don't  know 

1 

Did  not  respond 

15 

Does  have  an  SEPG 

8 

Also  did  not  answer  oompanion  question  (obstacles) 

5 

Did  answer  companion  question  (obstacles) 

10 

'  Based  on  responses  to  the  question:  *What  in  particular  has  facilitated  the  software  process  improvement 

effort  in  your  organization  and/or  division?' 

^  There  are  people  who  attended  the  workshops  and  filled  out  the  questionnaire  who  are  consultants  or  who 

come  from  organizations  that  do  not  themselves  do  software  development  work. 

Twelve  individuals 

indicated  in  response  to  either  this  question  and/br  the  one  about  obstacles  that  the  question  does  not  apply 

to  them  for  those  reasons,  and  they  are  excluded  from  the  figures  reported. 

^  Column  totals  to  more  than  100%  due  to  multiple  responses. 

*  Responses  include  reference  to  pre-existing  organization,  “pilot  project  success,* 

“proven  success...,' 

'nothing  in  particular,'  'core  competency  in  software  engineering  program,* 

'still  struggling  with  this,*  and 

unable  to  interpret  (4). 
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5  Characteristics  of  the  Organizations 


To  characterize  the  size  of  the  organizations  represented  at  the  SEPG  Workshop  the 
questionnaire  requested  information  regarding  the  number  of  individuals  working  in  software 
development  or  maintenance.  Most  of  the  workshop  participants  come  from  moderate  to  large 
sized  software  development  or  maintenance  organizations.  Note  in  Figure  9  that  three- 
quarters  of  the  participants  report  that  their  divisions  have  50  or  more  technical  staff  members, 
and  half  have  210  or  more.^  As  seen  In  Figure  10  on  the  next  page,  the  numbers  of 
managerial  staff  also  vary  considerably.  Three-quarters  of  the  organizations  have  1 0  or  more 
managerial  personnel,  half  have  at  least  25  managers,  and  one-fourth  have  at  least  56.^*^  In 
general,  there  is  roughly  a  ten  to  one  difference  between  the  numbers  of  technical  and 
managerial  personnel. 


Figure  9.  Size  of  the  Organization’s  Technical  Staff  (n=l37) 


Figure  11  includes  a  summary  of  the  number  of  software  development  or  maintenance 
projects  at  the  divisional  level.  As  seen  in  the  figure,  the  most  common  number  of  projects  is 
10.  The  median  number  of  projects  is  20.  and  the  largest  number  of  projects  reported  by  a 
workshop  participant  is  500  projects.  Note,  though,  that  one-fourth  of  the  participants  report 
coming  from  organizational  entities  that  supported  eight  or  fewer  projects. 

^  The  maximum  number  of  technical  staff  reported  is  20,000. 

^^e  maximum  number  of  managerial  personnel  reported  is  2,000. 
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Figure  10.  Size  of  the  Organization’s  Managerial  Staff  (n=l30) 


Figure  11.  Number  of  Software  Development  or  Maintenance  Projects  (n=133) 
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Tables  17  and  18  describe  the  amount  of  control  that  the  participants  report  their  divisions 
maintain  over  important  operationai  and  function^  areas.  Table  17  shows  that  78  percent 
claim  some  type  of  direct  control  over  the  establishment  of  standard  operating  procedures 
(SOPs).  Aimost  one  third  indicate  that  they  can  set  their  procedures  independently,  without 
required  higher  level  review.  On  the  other  hand,  as  seen  in  Table  18,  fewer  workshop 
participants  (56  percent)  report  any  type  of  control  over  the  funding  of  their  software 
improvement  efforts.  Only  21  percent  stated  that  they  had  complete  independence  in 
allocating  funding  for  process  improvement. 


Table  17.  Control  over  Standard  Operating  Procedures'*  (nsl69) 


Without  organizational  review 

Percent  responding 

32% 

Subject  to  organizational  approval 

46 

No  direct  control 

8 

Other  responses 

9 

Did  not  respond 

_5 

100% 

'  Based  on  responses  to  the  question:  ‘Does  your  division  have  direct  control  over  the  establishment  of  its 

standard  operating  procedures?* 

Table  18.  Control  over  Funding'*  (ns169) 


Without  organizational  review 

Percent  responding 

21% 

Subject  to  organizational  approval 

35 

No  direct  control 

31 

Other  responses 

5 

Did  not  respond 

_fi 

100% 

'  Based  on  responses  to  the  question;  “Does  your  division  have  direct  control  over  funding  for  its  software 
improvement  efforts?’ 
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6  Characteristics  of  the  Participants 
6.1  Background 

The  questionnaire  contains  several  questions  about  the  workshop  participants'  backgrounds 
in  software  engineering.  Job  titles  provide  one  way  to  characterize  that  background.  Notice 
in  Table  19  that  over  40  percent  of  the  participants  have  job  titles  that  imply  technical 
responsibilities;  about  half  of  these  individuals  (20  percent  of  all  of  the  participants)  identify 
themselves  specifically  as  “software  engineers.”  The  management  category  is  next  most 
common;  over  three  quarters  of  the  people  in  this  category  (22  percent  of  all  of  the 
participants)  identified  themselves  as  “manager.” 


Table  19.  Current  Job  Title'*  (ns169) 


Technical 

Percent  responding^ 
42% 

Management 

28 

Process 

15 

Other^ 

12 

Did  not  respond 

_5 

102% 

''  Based  on  responses  to  the  question;  ’What  is  your  current  job  title?’ 

^  The  responses  do  not  total  to  100%  due  to  rounding  error. 

^  Includes  specialist  (8),  consultant  (3),  operation  research  analyst  (2),  business  planning  (2),  and  resident 

1  affiliate  (1). 

_ 1 

Because  the  same  people  can  have  both  technical  and  managerial  responsibilities  and  job 
titles  alone  can  be  misleading,  we  also  asked  the  participants  to  classify  themselves  following 
the  categories  presented  in  Table  20.  The  participants  were  allowed  to  check  as  many 
classification  areas  as  they  felt  were  appropriate.  Notice  in  particular  that  almost  70  percent 
of  the  participants  chose  the  software  process  improvement  category  as  an  important 
descriptor  of  their  work  responsibilities. 

The  participants  were  also  asked  to  identify  the  highest  educational  degrees  they  have 
earned.  As  shown  in  Table  21.  almost  as  many  of  the  participants  hold  an  advanced 
(master’s  or  doctoral)  degree  as  have  an  associate’s  or  bachelor’s  degree.  Those  responding 
to  the  “other”  category  described  course  work  they  are  in  the  process  of  completing. 

The  participants  also  provided  information  regarding  the  major  fields  of  their  degrees.  The 
results  in  Table  22  show  that  most  of  the  participants  received  degrees  in  science  and 
engineering  related  fields.  However,  over  20  percent  have  management  degrees.  As  seen 
in  the  table,  there  is  a  good  deal  of  diversity  in  their  educational  backgrounds. 
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Table  20.  Self-Classification  of  Position^  (n=l69) 


Senior  managerial 

Middle  or  line  managerial 

27 

Senior  or  lead  technical 

34 

Other  technical 

6 

Software  process  improvement 

69 

Product  deveiopment 

10 

Software  quality  assurance 

13 

Configuration  management 

7 

Technical  support  or  training 

14 

Other 

11 

^  Based  on  responses  to  the  question:  “What  sort  of  a  position  is  it?' 

^  Column  totals  to  more  than  1 00%  due  to  multiple  responses. 

Table  21.  Highest  Degree  Earned^  (n=l69) 


Associate's 

Percent  responding 

2% 

Bacheior's 

46 

Master's 

40 

Doctorai 

6 

Other 

4 

Did  not  respond 

_2 

^  Based  on  responses  to  the  question: 

100% 

‘What  is  the  highest  degree  you  have  earned?' 

The  participants  were  asked  to  state  when  their  most  recent  degrees  were  awarded.  The 
dates  range  between  1 946  and  expected  in  1 994.  One-fourth  of  the  participants  received  their 
degrees  by  1971,  half  of  them  earned  their  most  recent  degree  by  1977,  and  three-quarters 
by  1983.  As  seen  in  the  following  section,  the  workshop  participants  have  a  substantial 
amount  of  software  employment  experience,  so  it  is  not  surprising  that  most  of  them 
completed  their  formal  education  some  time  ago. 


Table  22.  Major  Fieid^  (ns169) 


Mathematics  and  Science 
Mathematics 
Computer  Science 
Physics 
Other® 

Engineering 

Electrical^ 

Other® 

Business  and  Management 
Management® 

Information  Systems 
Other^ 

Other® 


Percent  responding^ 
51 


24 

20 

8 

1 


18 

15 

13 

8 

4 


30 

21 


8 


Did  not  respond 


2 


'  Based  on  responses  to  the  question:  “What  was  your  major  field?’ 

^  Both  columns  and  some  ’other*  categories  total  to  more  than  1 00%  due  to  multiple  responses.  Multiple 
responses  exist  for  some  individuals  with  double  majors  or  more  than  one  degree. 

^  Chemistry  (1),  and  Physical  Science  (1). 

*  Includes  one  ’electronics*  major. 

®  Computer  (5),  Software  (4),  Industrial  (4),  Aeronautical  (2),  Civil  (2),  Engineering  (2).  Mechanical  (2), 
Biomedical  (1),  Chemical  (1),  and  Structural  (1). 

^  Business  Administration  (6),  Business  (6),  MBA  (4),  Systems  Management  (3),  Engineering  Management 
(1),  Industrial  Management  (1),  Management  (1),  and  Management  Science  (1). 

^  Accounting  (3),  Ergonomics  (1),  Finance  (1),  Human  Resource  Development  (1),  Marketing  (1),  and 
Quantitative  Analysis  (1 ). 

®  Economics  (2),  Psychology  (2).  Sociology  (2),  Statistics  (2).  Anthropology  (1),  Architecture  (1),  Decision 
Sciences  (1),  Music  Education  (1),  Philosophy  of  Science  (1),  and  Political  Science  (1). 


6.2  Software  Experience 

The  workshop  participants  provided  information  regarding  their  experience  in  several  key 
areas.  They  were  asked  about  both  length  of  experience  in  years  and  breadth  of  experience 
based  on  the  number  of  software  projects  on  which  they  have  worked.  As  shown  in  Table  23, 
these  individuals  as  a  group  possess  a  good  deal  of  software  related  experience.  Their 
median  length  of  overall  software  experience  is  15  years.  Similarly,  half  of  the  participants 
have  worked  on  at  least  1 1  software  projects. 

The  participants  generally  have  spent  more  time  in  technical  than  in  managerial  positions. 
The  median  figures  for  software  technical  experience  are  10  years  and  6  or  7  projects. 
However,  the  participants’  median  software  management  experience  is  five  years  and  four 
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projects.  Given  the  relative  recency  of  software  process  improvement  efforts,  it  is  not 
surprising  that  they  have  less  experience  in  process  improvement  work.  The  median  number 
of  years  spent  on  process  improvement  is  two,  as  is  the  median  number  of  projects. 


Table  23.  Software  Experience'* 


Nui 

Present  organization 

nber  of  years 

Number  of  projects 

Median 

7.5 

5 

Number  responding 

166 

136 

Present  division 

Median 

4 

4 

Number  responding 

163 

130 

Present  position 

Median 

2 

2 

Number  responding 

165 

125 

Overall  software  experience 

Median 

15 

11 

Number  responding 

167 

135 

Software  technical  experience 

Median 

10 

6.5 

Number  responding 

162 

128 

Software  management  experience 

Median 

5 

4 

Number  responding 

155 

122 

Software  process  improvement  experience 

Median 

2 

2 

Number  responding 

162 

118 

'  Based  on  responses  to  the  question:  “What  is  your  experience  in...’ 
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7  Summary 


The  data  described  in  this  report  are  based  on  the  responses  to  a  self-administered 
questionnaire  received  from  169  individuals  who  participated  in  the  Software  Engineering 
Process  Group  (SEPG)  Workshop  held  in  Tysons  Comer,  Virginia,  in  April  of  1992.  These 
workshop  participants  constitute  approximately  70  percent  of  those  who  received  a  copy  of 
the  questionnaire.  The  questionnaire,  a  copy  of  which  is  appended  to  this  report,  asked  the 
participants  for  their  judgments  about  the  quality  of  the  workshop  and  for  information  about 
SEPGs  and  process  improvement  activities  in  their  home  organizations. 

Most  of  the  workshop  participants  come  from  moderate-  to  large-sized  software  development 
or  maintenance  organizations.  Three-quarters  of  the  participants  report  that  their  divisions 
have  50  or  more  technic^  staff;  half  have  210  or  more.  Half  of  the  participants  report  that 
their  divisions  currently  staff  at  least  20  projects;  one  quarter  have  40  or  more  projects 
underway. 

The  workshop  participants  bring  a  substantial  amount  of  software  experience,  having  both 
technical  and  managerial  experience.  On  average,  they  have  15  years  of  overall  software 
experience.  Their  software  process  improvement  experience  is  substantially  shorter,  on 
average  only  two  years. 

7.1  Evaluation  of  the  Workshop 

In  general,  the  participants  rated  the  SEPG  Workshop  highly.  They  were  particularly 
impressed  by  the  quality  of  the  keynote  speakers,  and  most  of  the  participants  (73  percent) 
indicated  that  they  would  attend  a  future  SEPG  Workshop.  Only  three  percent  said  they 
definitely  would  not  attend  another  SEPG  Workshop.  The  workshop  notebook,  however, 
received  less  favorable  reviews,  indicating  an  area  that  should  be  addressed  for  future  SEPG 
Workshops 

The  participants  expressed  similar  views  about  the  Capability  Maturity  Model  (CMM) 
Workshop  held  in  conjunction  with  the  SEPG  Workshop.  Half  of  those  responding  to  a 
question  about  the  CMM  disposition  reviews  chose  the  two  most  favorable  of  five  response 
categories.  Sixteen  percent  rated  the  reviews  as  “excellent”  while  only  eight  percent  called 
them  “inadequate.”  Moreover,  61  percent  of  those  responding  said  that  the  next  SEPG  and 
CMM  Workshops  should  also  be  held  concurrently. 

7.2  Status  of  SEPGs  and  Process  Improvement 

The  establishment  of  SEPGs  has  been  a  relatively  recent  and  rapidly  growing  phenomenon 
in  the  software  improvement  community.  Seventy-two  percent  of  the  workshop  participants 
(121  of  169  polled)  responded  that  they  come  from  organizations  that  maintain  SEPGs.  Most 
SEPGs  are  rather  new.  Few  were  established  prior  to  1990,  and  three-quarters  have  five  or 
fewer  full-time  staff  members. 
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The  workshop  participants  report  that  they  have  had  substantial  management  support  for  their 
efforts.  However,  there  has  been  less  commi^ent  among  the  technical  staff  thus  far. 

Most  SEPGs  have  focused  their  efforts  on  planning  oriented  activities.  However,  some 
workshop  participants  claim  that  their  SEPGs  have  already  instigated  changes  in  their 
organizations’  software  processes. 

Almost  two-thirds  of  the  workshop  participants  indicate  that  their  organizations  have  a 
software  process  improvement  plan  in  place.  Over  80  percent  of  those  who  have  a  process 
improvement  plan  base  it  on  SEI  methods. 

Finally,  the  workshop  participants  report  that  their  organizations  permit  a  substantial  degree 
of  management  autonomy  at  the  division  level;  78  percent  of  the  participants  report  some 
degree  of  division  level  control  over  standard  operating  procedures,  with  almost  a  third 
indicating  that  the  control  is  not  subject  to  further  organizational  review.  There  is  somewhat 
less  division  level  control  over  funding  for  software  improvement  efforts.  However,  56  percent 
of  the  participants  report  that  they  can  allocate  funds  subject  to  approval  by  the  organization, 
and  21  percent  can  approve  funding  without  required  organizational  review. 
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SEPG  Workshop: 
Questionnaire  &  Event  Evaluation 


April  9. 1992 


This  document  contains  questions  asking  for  your  judgements  about  this  week’s  SEPG  workshop. 
To  help  us  better  interpret  your  answers  about  the  woricshop,  the  document  also  contains 
questions  about  your  software  organization,  its  SEPG  and  your  own  software  experience.  Your 
answers  will  help  us  to  improve  future  workshops,  and  provide  useful  information  about  the 
functioning  of  SEPGs  in  general. 

In  order  to  ensure  meaningful  results,  everyone  who  is  attending  the  workshop  should  complete 
one  of  these  forms.  As  an  incentive  for  you  to  do  so,  everyone  who  completes  a  form  will  receive 
a  summary  report  based  on  the  data  we  collect. 

Please  read  and  answer  all  of  the  questions.  If  you  wish  to  comment  on  any  questions  or  qualify 
your  answers,  please  use  the  comment  spaces  provided.  We  will  be  using  optical  scanning 
technology  to  enter  the  data,  so  please  print  or  write  neatly  throughout  the  questionnaire. 

Your  answers  will  be  held  in  strict  confidence.  Any  information  identifying  you  will  be  kept 
separate  from  your  answers,  which  will  be  reported  in  combination  with  those  of  other 
individuals.  Specific  answers  will  not  be  identified  in  any  manner. 

Thank  you  for  your  help. 


Jonathan  Addelston,  PRC  Inc.,  McLean,  VA 

Miguel  A.  Carrio,  Jr.,  MTM  Engineering,  McLean,  VA 

Don  O'Neill,  Consultant,  Gaithersburg,  MD 

Stan  Rifkin,  Master  Systems,  Inc.,  McLean,  VA 

Joan  Weszka,  IBM  Federal  Sector  Division,  Bethesda,  MD 

SPIN 

Software  Process  Improvement  Network 
Washington,  D.C. 


Eugene  C.  Bounds 
Bill  Curtis 
Helen  Joyce 
Mike  Konrad 
Dave  McKeehan 
Mark  Paulk 
Rich  Pethia 

Software  Engineering  Institute 
Carnegie  Mellon  University 
Pittsburgh,  Pennsylvania 
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Part  I  Here  are  some  questions  about  your  background  in  software  engineering. 
1  What  is  your  current  job  title? 

1.1  What  sort  of  a  position  is  it?  (Mark  As  Many  Boxes  As  Apply) 

□  SENIOR  MANAGERIAL 

□  MIDDLE  OR  LINE  MANAGERIAL 

□  SENIOR  OR  LEAD  TECHNICAL 

□  OTHER  TECHNICAL 

□  SOFTWARE  PROCESS  IMPROVEMENT  (for  example,  in  an  SEPG) 

□  PRODUCT  DEVELOPMENT  OR  MAINTENANCE 

□  SOFTWARE  QUALITY  ASSURANCE 

□  CONFIGURATION  MANAGEMENT 

□  TECHNICAL  SUPPORT  OR  TRAINING 

□  OTHER  Please  Describe  Briefly 
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2  What  is  the  highest  degree  you  have  earned?  (Mark  One  Box) 

□  ASSOCIATES 

□  BACHELORS 

□  MASTERS 

□  DOCTORAL 


2.2 

When  was  your  most  recent 


degree  awarded?  (Please  Specify  [01  7R!3i‘^i^!b!7|8l‘?) 


3 


What  is  your  experience  in:  (Please  Specify  For  Each  Category) 
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Part  II  Here  are  some  questions  about  software  development/maintenance  in  your  organization. 


1  First  of  all,  does  your  organization  or  the  division  in  which  you  work  have  an  SEPG? 
(Mark  One  Box) 


□  NO- 

□  YES 


If  not,  skip  to  question  2  on  page  6 


(If  you  do  have  one) 

Where  in  the  organization  does  the  SEPG  exist?  (Mark  One  Box) 

□  THE  ORGANIZATION 

□  MY  DIVISION 

□  OTHER  Please  Describe  Briefly 


Who  in  the  organization  has  demonstrated  their  commitment  to  your  SEPG? 
(Mark  As  Many  Boxes  As  Apply) 

□  SENIOR  MANAGEMENT  (e.g.,  at  the  organization  or  division  level) 

□  MIDDLE  MANAGEMENT  (e.g.,  at  the  program  or  project  level) 

□  TECHNICAL  STAFF 

□  OTHERS  Please  Describe  Briefly 


When  was  the  SEPG  established?  (Please  Specify 


MONTH 


YEAR 


How  many  people  work  in  your  SEPG? 
(Please  Specify  For  Each  Category  101  -I 


I  »  I  >  I 
I  •  I  • 

EQ 


FULL  TIME 


PART  TIME 


CONSULTANTS 


Paged 


What  functions  does  your  SEPG  perform?  (Please  Describe  Briefly) 


What  is  your  SEPG’s  primary  res{)onsibility?  (Please  Describe  Briefly) 


What  kind  of  planning  and  related  activities  did  it  take  to  get  your  SEPG 
started?  (Please  Describe  Briefly) 


What  are  the  major  accomplishments  of  your  SEPG  thus  far? 
(Please  Describe  Briefly) 


+ 


2  Has  your  organization  developed  a  software  process  improvement  plan? 
(Mark  As  Many  Boxes  As  Apply) 

- □  NO 

-  □  YES  - 

-  □  YES  - 

-  □  YES  - 


zi 

Is  it  based  on:  (Mark  One  Box  For  Each  Category) 

!  DON’T 
YES  KNOW 

The  CMM  (CMU/SEI-91-TR-24andCMU/SEI-91-TR-25, 


released  at  the  1991  SEI  Affiliates  Symposium) .  □  □  □ 

Earlier  SEI  work  (CMU/SEI-87-TR-23) .  □  □  □ 

Other  process  improvement  work - ^ .  □  □  □ 

Please  Describe  Briefly 


3  What  major  obstacles  must  a  software  process  improvement  effort  overcome  in  your 
organization  and/or  division?  (Please  Describe  Briefly) 


FOR  THE  ORGANIZATION 
FOR  MY  DIVISION 

ELSEWHERE  - 

Please  Describe  Briefly 
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4  What  in  panicular  has  facilitated  the  software  process  improvement  effon  in  your 
organization  and/or  division?  (Please  Describe  Briefly) 


Approximately  how  many  people  were  primarily  engaged  in  software  development  or 
maintenance  efforts  in  your  division .  as  of  Mardi  31. 1992? 

(Please  Specify  For  Each  Category  101 


TECHNICAL  (for  example,  design,  analysis,  coding,  configuration 
management,  or  quality  assurance) 

MANAGERIAL  (including  people  with  some  technical  duties) 
OTHER  Please  Describe  Briefly 


6  Approximately  how  many  identifiable  projects  or  similar  product  development  or 
maintenance  grouns  did  vour  division  have  underway  as  of  March  31. 1992? 

(Please  Specify  j  :/12riSl7IgI^) 
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7  Does  your  division  have  direct  control  over  the  establishment  of  its  standard  operating 
procedures?  (Mark  One  Box) 

□  NO 

□  YES  -  SUBJECT  TO  APPROVAL  BY  THE  ORGANIZATION 

□  YES  --  WITHOUT  REQUIRED  REVIEW  BY  THE  ORGANIZATION 

□  OTHER 


8  Does  your  division  have  direct  control  over  funding  for  its  software  improvement  efforts? 
(Mark  One  Box) 

□  NO 

□  YES  - 

□  YES  - 

□  OTHER 


Part  in  Here  are  some  questions  about  this  and  future  SEPG  Workshops. 

1  How  do  you  rate  the  current  SEPG  Workshop?  (Mark  One  Box  For  Each  Category) 


INADEQUATE 

OK 

EXCELLENT 

The  Workshop  in  General . 

.  □ 

□ 

□ 

□ 

□ 

The  CMM  Workshop 

Disposition  Reviews . 

.  □ 

□ 

□ 

□ 

□ 

The  Keynote  Speakers . 

.  □ 

□ 

□ 

□ 

□ 

The  Panels . 

.  □ 

□ 

□ 

□ 

□ 

The  Working  Sessions . 

.  □ 

□ 

□ 

□ 

□ 

The  Workshop  Notebook . 

.  □ 

□ 

□ 

□ 

□ 

The  Birds  of  a  Feather  Sessions .... 

.  □ 

□ 

□ 

□ 

□ 

+ 


SUBJECT  TO  APPROVAL  BY  THE  ORGANIZATION 
WITHOUT  REQUIRED  REVIEW  BY  THE  ORGANIZATION 


Pages 


2  What  did  you  like  the  most  about  the  workshop?  (Please  Describe  Briefly) 


3  What  did  you  like  the  least  about  the  workshop?  (Please  Describe  Briefly) 


4  Should  future  SEPG  Workshops  continue  to  be  designed  around  Working  Sessions? 
(Mark  One  Box) 

□  YES 

□  NO 


□  I  DON’T  KNOW 


5  Would  you  attend  another  SEPG  workshop  in  the  future?  (Mark  One  Box) 


□  YES - 

□  NO 

□  DON’T  KNOW 


5.1 


When  should  it  be  held?  (Please  Specify  101 


MONTH  YEAR 

•  I  ■  I  /  I  '  I  • 


5.2 

Should  it  be  held  in  conjunction  with  another  CMM  Workshop? 
(Mark  One  Box) 

□  YES 

□  NO 

□  DON'T  KNOW 


6  What  would  you  change  for  future  Workshops?  (Please  Specify) 
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Part  rv 


Finally,  we  need  to  know  how  to  contact  you  to  give  you  your  summary  report 
comparing  SEPGs. 


This  information  is  for  administrative  use  oiilyT^ 
so  we  can  send  you  your  personalized  report. 
Your  identity  will  be  kept  in  strict  confidence. 

To  keep  your  responses  confidential,  we  will 
V^detach  this  sheet  prior  to  analyzing  the  data. 


Respondent  Identification:  (Please  Specify  (01  ■/I2l3l‘^l5lbl7i§l% 


YOUR  NAME: 


ADDRESS: 


TELEPHONE: 


/ 


EMAIL: 


Thank  you  very  much  for  your  time  and  effort!!! 
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